Systems, methods, and media for adjusting one or more images displayed to a viewer

ABSTRACT

Systems, methods, and media for adjusting one or more images displayed to a viewer are provided. In some implementations, the system comprises: at least one processing circuit configured to: detect an orientation of a viewer based on at least two points associated with the viewer; determine, based on the orientation, a rotational position of a first image to be displayed on a display; and generate a display image for display on the display, wherein the display image includes at least a portion of the first image rotated based on the rotational position.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part of U.S. patent applicationSer. No. 13/408,742, filed Feb. 29, 2012, which is hereby incorporatedby reference herein in its entirety.

BACKGROUND

The present disclosure relates generally to the field of displays. Morespecifically, the present disclosure relates to adjusting a displaybased on an orientation of a viewer.

Televisions and other viewing screens are usually configured with thevideo displayed on the screen horizontally in a way that is most suitedfor an upright (vertical) head display. While this display modeleverages the number of pixels available to the display, and is whattelevision viewers are accustomed to, it is not the most suitable forwhen viewers tilt their head (to the left or right, or to the back) orare not sitting upright (e.g., are laying on their side or are in areclined position). Viewers might be slouching on the coach or whensitting in a bed might rest their head on a pillow, thus making theimage display not optimal for viewing.

SUMMARY

Systems, methods, and media for adjusting one or more images displayedto a viewer are provided. In accordance with some implementations of thedisclosed subject matter, a system for adjusting one or more imagesdisplayed to a viewer is provided, the system comprising: at least oneprocessing circuit configured to: detect an orientation of a viewerbased on at least two points associated with the viewer; determine,based on the orientation, a rotational position of a first image to bedisplayed on a display; and generate a display image for display on thedisplay, wherein the display image includes at least a portion of thefirst image rotated based on the rotational position.

In accordance with some implementations of the disclosed subject matter,a method for adjusting one or more images displayed to a viewer isprovided, the method comprising: detecting, using a hardware processor,an orientation of a viewer based on at least two points associated withthe viewer; determining, based on the orientation, a rotational positionof a first image to be displayed on a display; and generating a displayimage for display on the display, wherein the display image includes atleast a portion of the first image rotated based on the rotationalposition.

In accordance with some implementations of the disclosed subject matter,non-transitory computer-readable media containing computer executableinstructions that, when executed by a processor, cause the processor toperform a method for adjusting video displayed to a viewer, the methodcomprising: detecting an orientation of a viewer based on at least twopoints associated with the viewer; determining, based on theorientation, a rotational position of a video image to be displayed on adisplay; and generating a display image for display on the display,wherein the display image includes at least a portion of the video imagerotated based on the rotational position.

BRIEF DESCRIPTION OF THE DRAWINGS

The details of one or more implementations of the subject matterdescribed in this specification are set forth in the accompanyingdrawings and the description below. Other features, aspects, andadvantages of the subject matter will become apparent from thedescription, the drawings, and the claims.

FIG. 1 is an environment view of a display and viewer in accordance witha described implementation.

FIGS. 2A and 2B are illustrations of a viewer's head and a method ofdetermining orientation of the viewer's head in accordance with adescribed implementation.

FIGS. 3A and 3B are illustrations of a rotated image on a display inaccordance with a described implementation.

FIG. 4 is a block diagram of a display system configured to adjust animage to display to a viewer in accordance with a describedimplementation.

FIG. 5 is a more detailed block diagram of the processing electronics ofthe display system of FIG. 4 in accordance with a describedimplementation.

FIG. 6 is a flow diagram of a process for adjusting an image to displayon a monitor using rotational position in accordance with a describedimplementation.

FIG. 7 is a flow diagram of a process for displaying an image andsupplemental data on a monitor in accordance with a describedimplementation.

FIG. 8 is a flow diagram of a process for adjusting an image to displayon a monitor using rotational position and transitions in accordancewith a described implementation.

DETAILED DESCRIPTION OF THE ILLUSTRATIVE IMPLEMENTATIONS

Referring generally to the figures, systems and methods for adjustingimages to display for a viewer are shown and described. The disclosurerelates to a system that determines the orientation of a viewer, such ashead position or the eye axis of a viewer. Based on the head positionand/or eye axis, the system adjusts an image on a display (e.g., atelevision or other monitor) to align with respect to the head positionand/or eye axis. In one implementation, the system adjusts the imagewithout mechanically moving the television or monitor housing, thoughthe housing may be also adjusted in alternative implementations. Thesystem keeps the image optimal for the position and/or orientation ofthe viewer's head rather than for a typical horizontal orientation. Inone implementation, the location and/or orientation of the eyes of theviewer is determined and the image display is adjusted based on thelocation and/or orientation of the eyes. Additionally, other data may beinferred from the head position and/or eye axis that can help determinea state or mindset of the viewer, and additional viewing parameters(e.g., image display settings such as brightness and contrast, soundsettings; auto off; etc.) may be adjusted accordingly.

The disclosure further relates to the display of supplemental data onthe display, which may supplement, for example, a video program, webpage, or other content being displayed. The supplemental data may bedisplayed in the space on the display that is no longer occupied by theimage after the adjusted image is displayed. For example, as an image isrotated 90 degrees, the aspect ratio of the image may be maintained,thereby leaving a portion of the screen beside the image that is blank.This portion may be filled with supplemental data, such as a programguide, controls for the content being played (e.g., stop, pause, volume,etc.), web content related to the content being played (e.g., blogs,wikis, messaging such as tweets, etc.), or other supplemental content.

Referring now to FIG. 1, an environment 100 in which the systems andmethods of the present disclosure may be applied is shown. Environment100 is provided as an example of a display and viewer configuration in,for example, a room; it should be understood that environment 100including a display and viewer may be implemented in any type of settingwithout departing from the scope of the present disclosure.

Environment 100 is shown to include a display device 102 and a viewer104 watching display 102. Display 102 is shown as part of a televisionin the implementation shown in FIG. 1. According to otherimplementations, display 102 may be a display for any type of electronicdevice (e.g., television, personal computer (PC), laptop, etc.) or otherelectronic source (e.g., video game console, etc.). Display 102 isconfigured to provide a display for viewer 104.

A viewer orientation detection device 106 is shown coupled to atelevision set-top box 108. Viewer orientation detection device 106 maybe any device configured to determine the orientation (e.g., a headposition or eye axis) of a viewer of display 102. In one implementation,viewer orientation detection device 106 comprises a camera. While thesystems and methods of the present disclosure describe viewerorientation detection device 106 as a camera, it should be understoodthat in other implementations, viewer orientation detection device 106may comprise any detection device configured to detect at least one ofposition and orientation of at least a portion of viewer 104, or may beany type of sensor configured to detect a location of viewer 104 orviewer 104 behavior. For example, according to another implementation,the viewer orientation detection device may be a laser-based detector oran infrared detector configured to detect at least one of position andorientation of at least a portion of a viewer. Additionally, accordingto another implementation, the viewer orientation detection device maybe used in combination with a device worn by the viewer (e.g., a laseremitting apparatus) that can communicate with the viewer orientationdetection device. Set-top box 108 receives data from viewer orientationdetection device 106 for determining the orientation of viewer 104.Viewer orientation detection device 106 may further be coupled todisplay 102, a remote source, or any other device configured to applyrotational position adjustments to an image for display on display 102according to various implementations.

In the implementation of FIG. 1, since display 102 is a televisiondisplay, a television set-top box 108 is shown and is configured toprovide the television with content to display. In otherimplementations, display 102 may be a display for another electronicdevice (e.g., monitor) and set-top box 108 may not be included inenvironment 100.

Viewer 104 is shown watching display 102 while laying on a seat or couch110. Viewer 104 may be watching display 102 while his or her head istilted at an angle from display 102. This may be the result of layingdown on a seat or sofa, as a result of sitting or standing, or a resultof any other type of motion or viewing position. Display 102 istraditionally configured to provide a display for viewer 104 assumingthe viewer is in an upright position (e.g., eye axis generally parallelto a horizontal dimension of the monitor). As viewer 104 watches thecontent on display 102 when his or her head is tilted at an angle, thedisplay is no longer optimal for viewer 104.

Referring now to FIGS. 2A and 2B, viewer orientation detection device106 can determine a position of the head of viewer 104 and the eye axisof viewer 104. Determining the position of the head of viewer 104 mayinclude determining the location of the head. For example, the headlocation may be represented as coordinates in a three-dimensional (3D)space (e.g., (x, y, z) coordinates, polar coordinates, cylindricalcoordinates, etc.) or two-dimensional (2D) space (e.g., (x, y)coordinates), may be represented as an absolute distance from thedisplay 102, may be represented as a distance and angular distance fromthe monitor (e.g., an absolute distance from the display 102 and anangle between the viewer 104 and display 102), etc. Further, theorientation of the head may be determined. The orientation may includedetermining if the head is turned, if the head is directly looking atthe monitor or not, or otherwise.

When the position of the head is determined, the eye axis of viewer 104may be determined. The eyes of the head may be located and the eye axismay be defined as the axis line that connects the locations of the twoeyes in the 2D or 3D space (e.g., in (x, y) or (x, y, z) coordinates).Determining the eye axis of viewer 104 may include any number of eyedetection techniques (such as, e.g., those used in conjunction withfacial recognition techniques or software). For example, light may bereflected from the eyes of viewer 104 and sensed by a camera of viewerorientation detection device 106, and the information from the changesin reflection are used to analyze the eye axis and other eye features.It should be understood the eye detection methods may vary according tovarious implementations.

As one example, in FIG. 2A, viewer 200 may be viewing display 102 in anupright horizontal position. The eye location 204 and eye axis 202 ofthe viewer may be determined. In FIG. 2A, eye axis 202 is shown as ahorizontal line, indicating to monitor 102 that the viewer is in anupright horizontal position and that no display adjustment is needed.Eye axis 202 is a line with the same coordinate values in two directions(e.g., the y- and z-coordinates for both eyes are the same, indicatingboth eyes are at the same depth and relative height from display 102),while the x-coordinates of the eyes are different (which isrepresentative of the spacing between the eyes).

As another example, in FIG. 2B, viewer 250 is shown at an angle (e.g.,angle A) that is offset from the horizontal axis (e.g., x-axis) relativeto display 102. Viewer 250 may be laying down or otherwise tilting hisor her head to watch display 102. The head position may be determined bya processing circuit within viewer orientation detection device 106 orset-top box 108, including the orientation of the head. In FIG. 2B, thehead of viewer 250 is pointing towards a location up and to the left ofa “center” of the display screen, but the viewer may be looking forwardto watch display 102.

The eye locations 253, 254 and/or eye axis 252 of viewer 250 may bedetermined. Since the head of viewer 250 is at an angle that is offsetfrom the horizontal axis relative to display 102, eye axis 252 may be aline, each point of which has different coordinate values in threedirections (on the (x, y, z) axis), indicating the distance of each eyeto display 102 is different. Further, one eye 254 may be “higher” in ay-axis dimension than the other eye 253 as shown, further changing theperspective of the view of viewer 250. Using eye axis 252, a rotationalposition of an image on display 102 may be determined that correlateswith eye axis 252. The adjustment of the image is described in greaterdetail in FIGS. 4-8. One or more of the coordinate values of each eyemay be disregarded; for example, the fact that eye 254 is further frommonitor than eye 253 may be immaterial to a determination that eye axis252 is at an angle A relative to display 102. The system may beconfigured to determine the angle A and adjust the image orientation ondisplay 102 based on angle A.

In one implementation, portions of the head of the viewer may be used toassist in the head position and/or eye axis determinations. For example,a location of a forehead or chin of the viewer may be determined, andthe head position and eye axis locations may be determined based on thelocation of the forehead or chin. The forehead or chin may further beused to determine if the viewer is leaning forwards or backwards. Forexample, a forehead farther away from the monitor than the chinindicates that the viewer is leaning backwards.

Referring now to FIGS. 3A and 3B, the result of a rotational positionadjustment of an image on a monitor is shown in greater detail. In FIG.3A, an image 302 is shown on display 300 unadjusted. When a viewer iswatching display 300 in an upright position (e.g., viewer 200), image302 is unadjusted and there is no unused space on display 300. In otherimplementations, image 352 may be tilted and rotated, just rotated, orany combination of tilted, rotated, stretched, or any other type ofmodification. For example, image 352 may only be rotated if the vieweris to one side of the display, image 352 may be stretched as well if theviewer's head is not pointing at the display, image 352 may be tilted aswell if the viewer's head is above or below the display, etc.

In FIG. 3B, image 352 is shown tilted (e.g., rotated, adjusted, etc.)and stretched as a result of a head position and eye axis of a viewersuch as viewer 250. Image 352 is adjusted such that the plane of image352 is compatible with the eye axis of the viewer (e.g., the rotationalposition of image 352 is roughly parallel with the eye axis).

In FIG. 3B, image 352 is shown with a border to define the area ofdisplay 350 in which the image is displayed; the border may or may notbe visible to the viewer according to various implementations. Whenimage 352 is rotated, some of the display of display 350 becomes unused.According to one implementation, the unused space may be used to displaysupplemental data 354. Supplemental data 354 may include the name of theprogram and/or program information (e.g., air times, channelinformation, etc.), controls for the display (e.g., the viewer canselect, via a television remote, to pause or stop the display), and/orweb content related to the display (e.g., a link that a viewer canselect to access reviews, fan sites, blogs, posts, or other webcontent), etc.

Referring now to FIG. 4, a block diagram of a display system is shown inaccordance with a described implementation. The display system includesa viewer 402, a viewer orientation detection device 404 configured todetect the orientation of viewer 402, content 406 (e.g., a televisionprogram) from a content source (e.g., a remote internet server, a cableTV source, a satellite TV source, a DVD player, etc.), a televisionset-top box 408, and a display 414 configured to display content 406.Viewer 402 may be the viewer as generally described in FIGS. 1-3.Display 414 may be a display for a television in the implementation ofFIG. 4, and set-top box 408 is configured to be connected to thetelevision and to provide display 414 with adjusted content 406. Itshould be understood that in other implementations, display 414 may notbe a television display and the display system may not include a set-topbox 408. In such an implementation, the components of set-top box 408may be a part of display 414 or part of another device.

Viewer orientation detection device 404 may be any device configured tocapture viewer 402 properties such as the head position or eye axis ofviewer 402. In one implementation, viewer orientation detection device404 comprises a camera. The camera is configured to capture images ofviewer 402 and to provide the images to the processing electronics 412of set-top box 408 for processing. In one implementation, viewerorientation detection device 404 may be a stationary camera with a fixedlocation, and may capture viewer 402 properties when the viewer is inrange of the camera. In another implementation, viewer orientationdetection device 404 may be a camera configured to pan, tilt, or zoom inor out in order to capture an image of viewer 402. In someimplementations, viewer orientation detection device 404 may determinean orientation and/or position of viewer 402 as the viewer moves aboutenvironment 100, for example, by moving (e.g., panning, tilting, zoomingin or out, and/or any other suitable type of movement) in order tocapture an image of the viewer.

In some implementations, any images captured by viewer orientationdetection device 404 may be only used to determine the propertiesdescribed herein and used for no other purpose. These images may bedeleted from all components of FIGS. 4 and 5 shortly after theproperties are determined in some implementations.

Content 406 may be any content to be displayed on display 414. Forexample, display 414 may be part of a television and content 406 mayinclude a television program from any source (e.g., cable television,satellite television), a movie being played via a DVD player, a videogame, internet content, etc. It should be understood that the type ofcontent 406 displayed on display 414 may be varied without departingfrom the scope of the present disclosure.

Television set-top box 408 is configured to receive content 406 todisplay on display 414 via input/output (I/O) interface 410. Set-top box408 is further configured to receive images of viewer 402 and/or otherdata from viewer orientation detection device 404 via I/O interface 410.I/O interface 410 is configured to facilitate communications, either viaa wired connection or wirelessly, with viewer orientation detectiondevice 404 and the providers of content 406.

Set-top box 408 receives images and other data from viewer orientationdetection device 404 and provides the images and data to processingelectronics 412 (shown in greater detail in FIG. 5). Processingelectronics 412 are configured to use the images and other data todetermine an orientation of viewer 402. Using the orientation of viewer402 and the received content 406, processing electronics 412 areconfigured to determine a rotational position of images of the content.Processing electronics 412 provides display 414 with the rotationalposition to allow display 414 to optimally display content 406 ondisplay 414.

Referring now to FIG. 5, processing electronics 412 is shown in greaterdetail. Processing electronics 412 is shown to include a processor 502and memory 504. Processor 502 may be implemented as one or more generalpurpose processors, application specific integrated circuits (ASIC),field programmable gate arrays (FPGAs), a group of processingcomponents, or other suitable electronic processing components in one ormore housings, local and/or remote from display 102. Memory 504 maycomprise one or more devices (e.g., RAM, ROM, Flash memory, hard diskstorage, etc.) for storing data and/or computer code for completingand/or facilitating the various processes described herein. Memory 504may be or include non-transient volatile memory or non-volatile memory.Memory 504 may include data base components, object code components,script components, or any other type of information structure forsupporting the various activities and information structures describedherein. Memory 504 may be communicably connected to processor 502 andincludes computer code or instructions for executing one or moreprocesses described herein.

Memory 504 may include various modules for completing the methodsdescribed herein. It should be understood that memory 504 may includemore or less modules, and that some of the activity described asoccurring within memory 504 and processing electronics 502 may becompleted by modules located remotely from processing electronics 412.For example, some or all of the viewer orientation information may bedetermined by viewer orientation detection device 404 instead ofprocessing electronics 502.

Memory 504 includes a viewer detection module shown in FIG. 5 as facialrecognition module 506. Facial recognition module 506 is configured toreceive images and data from viewer orientation detection device 404 andto identify viewer 402 in the images. Facial recognition module 506 mayfurther be configured to determine the orientation of viewer 402 (e.g.,the head position and/or eye axis of viewer 402). Facial recognitionmodule 506 may include logic for determining the head position and eyeaxis as generally described in FIGS. 2A and B. According to otherimplementations, the process of identifying viewer 402 and determiningthe orientation of viewer 402 may occur in separate modules.

Memory 504 may further include display configuration module 508. Displayconfiguration module 508 is configured to use the orientation of viewer402 (determined by facial recognition module 506 or another module) todetermine a display configuration for the content to be displayed. Thedisplay configuration includes a rotational position of the images to bedisplayed on the display. For example, based on an eye axis, displayconfiguration module 508 is configured to determine a rotationalposition of the image to match the eye axis.

Display configuration module 508 may include logic for determiningrotational positions for an image or for predetermined rotationalpositions. For example, based on the eye axis of viewer 402, arotational position for an image is determined. In one implementation,the offset of the eye axis is used to determine the rotational position.For example, if a viewer is viewing display 414 at a 20 degree angle tothe left of a vertical axis of display 414, the image may be rotated tothe left at a 20 degree angle to compensate. The rotational position ofthe image may include rotating the image left or right based on ifviewer 402 is to the left of center or right of center of display 414.The rotational position of the image may include rotating the image upor down based on if the eye axis of viewer 402 is above the center orbelow the center of display 414. These viewer 402 positions are shown ingreater detail in FIGS. 2A and 2B.

Further, the rotational position of the image may include rotating theimage based on a tilt of the head of viewer 402. For example, the headmay be tilted to one side or the other, or tilted backward or forward.If the head is tilted to one side, the image may be rotated to the sidethe head is pointing towards. If the head is tilted back, the image maybe tilted forward. In one implementation, chin or forehead locationinformation (or other location information for portions of the viewer402's head) may be used to determine whether to tilt an image forward orbackward.

In one implementation, instead of an exact calculation of a rotationalposition based on an eye axis, there may be a predetermined number ofpreset rotational positions. For example, an image to be displayed maybe rotated at 15 degree increments based on the eye axis of viewer 402.If the eye axis is offset by, for example, 10 degrees, the image may berotated by 15 degrees, but if the eye axis is offset by, for example, 5degrees, the image may not be rotated at all. The increments by whichthe image is rotated may be 5 degrees, 15 degrees, 30 degrees, or anyother increment. These increments may be used for rotating the imageleft, right, up, or down, which may allow for a finite number ofpossible rotational positions, according to one implementation.

In one implementation, display configuration module 508 may determinethe zoom at which an image is shown. According to one implementation,when a rotational position of an image is changed, there may be unusedspace on display 414. Therefore, the image may then be scaled (e.g.,zoomed in) such that a maximum amount of space in display 414 may beused.

In another implementation, the zoom may be based on viewer proximity.Viewer proximity information (e.g., distance to the display) may beprovided to display configuration module 508 and used to determine howmuch to zoom in or out on the image shown in display 414. For example,if viewer 402 is close to the display, the image may be zoomed out on.

Display configuration module 508 may include any number of thresholdvalues relating to rotational positions of an image. For example, therotation of an image or other display properties may be restricted basedon display settings or viewer settings. If a viewer is far away fromdisplay 414, there may be a limit to the zooming in on the image. If aviewer is too offset to display 414 (too far to one side of thedisplay), there may be a limit to the amount of rotation of the image.Display configuration module 508 may otherwise crop the image in variousways, according to various implementations.

Memory 504 may further include image rotation module 510. Using thedisplay configuration (determined by display configuration module 508),image rotation module 510 applies rotations, reflections, and tilts tothe image. In other words, image rotation module 510 applies therotational positions to the image for display. Image rotation module 510receives the content 406 to display, applies the rotational positions tothe content 406, and provides the adjusted content to display 414 fordisplay. In various implementations, the activities of module 508 and510 may be performed in the same module.

Memory 504 may further include transition module 512. Transition module512 is configured to control a transition of the display from oneconfiguration to a new configuration based on the new rotationalposition of the image. Instead of instantaneously changing therotational position of an image on the display, it may be desired toslowly transition an image from the traditional horizontal display toanother display. For example, if viewer 402 suddenly changes his or herorientation, it may be desirable not to immediately change display 414,since viewer 402 may further change his or her orientation. In such acase, there may be a built-in delay, filter, or hysteresis beforechanging the rotational position of the image on display 414. Forexample, transition module 512 may set a five-second delay before anyrotational position adjustment is applied to an image. As otherexamples, a delay of less than about five seconds, less than about twoseconds, less than about ten seconds, or any other length delay, may beapplied.

Transition module 512 may further control the rotational position changeof the image on display 414. For example, an orientation of viewer 402may suggest that an image should be rotated 60 degrees to the left.However, it may not be desirable to suddenly change from a un-rotatedimage to an image rotated 60 degrees to the left. Therefore, in oneimplementation, a transition may be applied to the image such that it iscontinually rotated slowly into the final position of 60 degrees to theleft. For example, the image may be rotated at a rate of about 5 degreesper second or less, about 15 degrees per second or less, etc. In anotherimplementation, instead of a continuous rotation, the rotation may beperformed in “stages”. For example, the image may be rotated 5 degreesat a time, up to the final desired rotation of 60 degrees. As anotherexample, the image may be rotated 10 degrees every second until thefinal desired rotation of 60 degrees is reached. Transition module 512may be configured to determine a rotational position pattern to apply tothe image given the information about the desired final rotationalposition of the image from modules 508, 510. Transition module 512 mayprovide the rotational position pattern to image rotation module 510,and image rotation module 510 may adjust the images accordingly.

Memory 504 further includes viewer attention module 514. Viewerattention module 514 receives the images and other data from viewerorientation detection device 404 and is configured to determine whetherviewer 402 is watching the display. This determination may then be usedto alter the display of images on display 414. As one example, if aviewer 402's eyes are closed, viewer attention module 514 may determinethat the viewer 402's eyes are closed using the images and other datafrom viewer orientation detection device 404. Viewer attention module514 may then determine a length of time the viewer 402's eyes are closedand may either turn off display 414, pause display 414, or take someother display-related action in response to the viewer not watching thedisplay. As another example, when the viewer 402 begins viewing themonitor again, display 414 may be unpaused when viewer attention module514 detects viewer 402 once again watching the monitor.

Memory 504 further includes supplemental data module 516. Supplementaldata module 516 may receive or determine supplemental data that may bedisplayed on display 414. Supplemental data may include a program guideor program guide information (e.g., title of a television program, airtimes of the program, short description of the program, actors in theprogram, current channel of the television, etc.), controls for thedisplay (e.g., a pause button or stop button that viewer 402 may selectvia remote control, a volume control, etc.), or web content relateddirectly or indirectly to the image being displayed (e.g., a link to awebsite, a blog post, a tweet, a post or message on a social networkingsite or other site, or other content selectable by viewer 402 via remotecontrol). Supplemental data may further include any other data notrelated to the image being displayed (e.g., time, weather, etc.).

Referring generally to FIGS. 6-8, various processes that may be executedby the modules of processing electronics 412 are described. It should beunderstood that the various blocks of FIGS. 6-8 may be executed in anyorder or in any combination in accordance with any number ofimplementations.

Referring now to FIG. 6, a flow diagram is shown of a process 600 foradjusting an image to display on a monitor using rotational position inaccordance with a described implementation. Process 600 includesdetecting orientation of a viewer (block 602) of a display. A viewerorientation detection device may be used to capture images of theviewer, and a facial recognition module (e.g., facial recognition module506 of FIG. 5) or another device may determine the orientation of theviewer as described in the present disclosure. The orientation of theviewer may include head position, eye axis information, or other datarelating to a viewer location or position.

Process 600 further includes using the orientation of a viewer todetermine a rotational position of a image to be displayed on a displayto the viewer (block 604). The rotational position of the image may bedetermined such that the viewer may view the image optimally. Arotational position may be determined by, for example, the displayconfiguration module 508 of FIG. 5. Process 600 further includesdisplaying the image to the viewer on the display based on therotational position (block 606).

Referring now to FIG. 7, a flow diagram of a process 700 for displayingan image and supplemental data on a monitor is shown in accordance witha described implementation. Process 700 includes displaying a rotatedimage on a display (block 702). When the rotated image is displayed,there may be areas of the display that are unused. Process 700 includesdetermining an area of the display not occupied by the rotated image(block 704).

Process 700 further includes determining supplemental data to display inthe vacant area of the display determined in block 704 (block 706).Supplemental data may include a program guide or program guideinformation (e.g., title of a television program, air times of theprogram, short description of the program, actors in the program,current channel of the television, etc.), controls for the display(e.g., a pause button or stop button that viewer 402 may select viaremote control, a volume control, etc.), and/or web content related tothe image being displayed (e.g., a link to a website, a blog post, atweet, a post or message on a social networking site or other site, orother content selectable by viewer 402 via remote control). Supplementaldata may be determined by, for example, supplemental data module 516 ofFIG. 5. Supplemental data maybe viewer configurable (that is, the viewerchooses what supplemental data the viewer sees).

Process 700 includes displaying the supplemental data in addition to therotated image (block 708). The activities of block 708 may includedetermining a display format for the supplemental data (e.g., font size,which portions of the vacant display to use, etc.).

Referring now to FIG. 8, a flow diagram of a process 800 for adjustingan image to display on a monitor using rotational position andtransitions is shown, in accordance with a described implementation.Process 800 includes detecting orientation of a viewer via a viewerorientation detection device (block 802) and determining rotationalposition of an image to be displayed on a display based on theorientation (block 804). Process 800 further includes determining atransition of the image (block 806). Block 806 may include determininghow to transition from the image from one position on the display toanother. Transitions may include, for example, a sudden rotationalposition change of the image, a built-in delay configured to keep theimage in a first position for a specified amount of time beforetransitioning the image, a gradual transition from one position of animage to another, etc. Block 806 may be executed via, for example,transition module 512 of FIG. 5. Process 800 further includes displayingthe image to the viewer on the display based on the rotational positionand the transitions (block 808).

In one implementation of the present disclosure, in addition todetermining an orientation of a particular viewer, the systems andmethods of the present disclosure may be used to identify one or moreparticular viewers whose orientations are to be determined. For example,if multiple people are viewing a monitor, logic for identifying aparticular viewer whose orientation is to be determined may be included.For example, the orientation of a main viewer (e.g., the owner of themonitor, a viewer determined to be logged-in to a user accountassociated with the monitor and/or a set-top box, a viewer determined tobe closest to a user device such as a mobile phone that is associatedwith the monitor, and/or any other suitable main viewer) may bedetermined, or the orientations of multiple viewers may be determined.The viewer orientation detection device may provide images and otherdata for multiple viewers, and a rotational position of an image on thedisplay may be determined such that the display is optimal for just oneviewer, or is optimal for all viewers, or for any combination ofviewers. The process may include any number of viewer identificationtechniques to identify the particular viewer(s) whose orientations areto be determined.

It should be understood that while the processing electronics used toexecute the systems and methods of the present disclosure is shown localto the monitor, the processing electronics may otherwise be locatedremotely from the monitor in other implementations.

In various implementations, the modules described herein may be used inother television or display-based systems. For example, viewer attentionmodule 514 of FIG. 5 may be used in other display-based systems. In oneimplementation, viewer attention module 514 information may be used tochange any aspect of the display (e.g., volume, brightness, contrast,etc.) for any type of content being displayed.

Configurations of Various Illustrative Implementations

Implementations of the subject matter and the operations described inthis specification may be implemented in digital electronic circuitry,or in computer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them. Implementations of the subjectmatter described in this specification may be implemented as one or morecomputer programs, i.e., one or more modules of computer programinstructions, encoded on one or more computer storage medium forexecution by, or to control the operation of, data processing apparatus.Alternatively or in addition, the program instructions may be encoded onan artificially-generated propagated signal (e.g., a machine-generatedelectrical, optical, or electromagnetic signal) that is generated toencode information for transmission to suitable receiver apparatus forexecution by a data processing apparatus. A computer storage medium maybe, or be included in, a computer-readable storage device, acomputer-readable storage substrate, a random or serial access memoryarray or device, or a combination of one or more of them. Moreover,while a computer storage medium is not a propagated signal, a computerstorage medium may be a source or destination of computer programinstructions encoded in an artificially-generated propagated signal. Thecomputer storage medium may also be, or be included in, one or moreseparate components or media (e.g., multiple CDs, disks, or otherstorage devices). Accordingly, the computer storage medium is bothtangible and non-transitory.

The operations described in this disclosure may be implemented asoperations performed by a data processing apparatus on data stored onone or more computer-readable storage devices or received from othersources.

The systems and methods of the present disclosure may be completed byany computer program. A computer program (also known as a program,software, software application, script, or code) may be written in anyform of programming language, including compiled or interpretedlanguages, declarative or procedural languages, and it may be deployedin any form, including as a stand-alone program or as a module,component, subroutine, object, or other unit suitable for use in acomputing environment. A computer program may, but need not, correspondto a file in a file system. A program may be stored in a portion of afile that holds other programs or data (e.g., one or more scripts storedin a markup language document), in a single file dedicated to theprogram in question, or in multiple coordinated files (e.g., files thatstore one or more modules, sub-programs, or portions of code). Acomputer program may be deployed to be executed on one computer or onmultiple computers that are located at one site or distributed acrossmultiple sites and interconnected by a communication network.

The processes and logic flows described in this specification may beperformed by one or more programmable processors executing one or morecomputer programs to perform actions by operating on input data andgenerating output. The processes and logic flows may also be performedby, and apparatus may also be implemented as, special purpose logiccircuitry (e.g., an FPGA or an ASIC).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random access memory or both. The essential elements of a computer area processor for performing actions in accordance with instructions andone or more memory devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom or transfer data to, or both, one or more mass storage devices forstoring data (e.g., magnetic, magneto-optical disks, or optical disks).However, a computer need not have such devices. Moreover, a computer maybe embedded in another device (e.g., a mobile telephone, a personaldigital assistant (PDA), a mobile audio or video player, a game console,a Global Positioning System (GPS) receiver, or a portable storage device(e.g., a universal serial bus (USB) flash drive), etc.). Devicessuitable for storing computer program instructions and data include allforms of non-volatile memory, media and memory devices, including by wayof example semiconductor memory devices (e.g., EPROM, EEPROM, and flashmemory devices; magnetic disks, e.g., internal hard disks or removabledisks; magneto-optical disks; and CD-ROM and DVD-ROM disks). Theprocessor and the memory may be supplemented by, or incorporated in,special purpose logic circuitry.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anydisclosures or of what may be claimed, but rather as descriptions offeatures specific to particular implementations of particulardisclosures. Certain features that are described in this disclosure inthe context of separate implementations may also be implemented incombination in a single implementation. Conversely, various featuresthat are described in the context of a single implementation may also beimplemented in multiple implementations separately or in any suitablesubcombination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination may in some cases be excisedfrom the combination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemsmay generally be integrated together in a single software product orpackaged into multiple software products.

The features disclosed herein may be implemented on a smart televisionmodule (or connected television module, hybrid television module, etc.),which may include a processing circuit configured to integrate internetconnectivity with more traditional television programming sources (e.g.,received via cable, satellite, over-the-air, or other signals). Thesmart television module may be physically incorporated into a televisionset or may include a separate device such as a set-top box, Blu-ray orother digital media player, game console, hotel television system, andother companion device. A smart television module may be configured toallow viewers to search and find videos, movies, photos and othercontent on the web, on a local cable TV channel, on a satellite TVchannel, or stored on a local hard drive. A set-top box (STB) or set-topunit (STU) may include an information appliance device that may containa tuner and connect to a television set and an external source ofsignal, turning the signal into content which is then displayed on thetelevision screen or other display device. A smart television module maybe configured to provide a home screen or top level screen includingicons for a plurality of different applications, such as a web browserand a plurality of streaming media services (e.g., Netflix, Vudu, Hulu,etc.), a connected cable or satellite media source, other web“channels”, etc. The smart television module may further be configuredto provide an electronic programming guide to the viewer. A companionapplication to the smart television module may be operable on a mobilecomputing device to provide additional information about availableprograms to a viewer, to allow the viewer to control the smarttelevision module, etc. In alternate implementations, the features maybe implemented on a laptop computer or other personal computer, asmartphone, other mobile phone, handheld computer, a tablet PC, or othercomputing device.

In situations in which the systems described herein collect personalinformation about and/or images of viewers, or make use of personalinformation and/or images of viewers, the viewers may be provided withan opportunity to control whether programs or features collect viewerinformation (e.g., information about a viewer's social network, socialactions or activities, profession, a viewer's preferences, or a viewer'scurrent location) and/or images. In addition, certain data may betreated in one or more ways before it is stored or used, so thatpersonal information and/or images are removed. For example, a viewer'sidentity may be treated so that no personally identifiable informationcan be determined for the viewer, or a viewer's geographic location maybe generalized where location information is obtained (such as to acity, ZIP code, or state level), so that a particular location of aviewer cannot be determined. Thus, the viewer can have control over howinformation and/or images about the viewer are collected and used by acontent server. As another example, images may be masked when capturedso that the identity, facial expression, gender, and any othercharacteristics of the viewer other than position and/or orientation arenot determined from the images.

Thus, particular implementations of the subject matter have beendescribed. Other implementations are within the scope of the followingclaims. In some cases, the actions recited in the claims may beperformed in a different order and still achieve desirable results. Inaddition, the processes depicted in the accompanying figures do notnecessarily require the particular order shown, or sequential order, toachieve desirable results. In certain implementations, multitasking andparallel processing may be advantageous.

What is claimed is:
 1. A system for adjusting one or more imagesdisplayed to a viewer, comprising: at least one processing circuitconfigured to: detect an orientation of a viewer to a display based onat least two points associated with the viewer; determine, based on theorientation of the viewer to the display, a rotation of a first image tobe displayed on the display; determine a rotational limit for therotation of the first image; generate a display image for display on thedisplay, wherein the display image includes at least a portion of thefirst image rotated based on the rotation of the first image and subjectto the rotational limit; determine a portion of the display that doesnot include the portion of the first image rotated based on the rotationof the first image; identify supplemental content to be presented in theportion of the display that does not include the portion of the firstimage rotated based on the rotation of the first image; and cause theidentified supplemental content to be presented in the portion of thedisplay that does not include the portion of the first image rotatedbased on the rotation of the first image.
 2. The system of claim 1,wherein the at least one processing circuit is further configured todetect an eye axis of the viewer.
 3. The system of claim 1, furthercomprising a camera that detects an image of the viewer.
 4. The systemof claim 1, wherein the at least one processing circuit is furtherconfigured to: increment the rotation of the first image to a newrotation of the first image; and generate another display image thatincludes the at least a portion of the first image or a subsequent imagerotated based on the new rotation of the first image.
 5. The system ofclaim 1, wherein the at least one processing circuit is furtherconfigured to delay generating the display image a predetermined amountof time after detecting a change in the orientation of the viewer. 6.The system of claim 1, wherein the supplemental data comprises at leastone of a program guide, controls for the at least a portion of the firstimage, and web content related directly or indirectly to the firstimage.
 7. A method for adjusting one or more images displayed to aviewer, comprising: detecting, using a hardware processor, anorientation of a viewer to a display based on at least two pointsassociated with the viewer; determining, based on the orientation of theviewer to the display, a rotation of a first image to be displayed onthe display; determining a rotational limit for the rotation of thefirst image; generating a display image for display on the display,wherein the display image includes at least a portion of the first imagerotated based on the rotation of the first image and subject to therotational limit; determining a portion of the display that does notinclude the portion of the first image rotated based on the rotation ofthe first image; identifying supplemental content to be presented in theportion of the display that does not include the portion of the firstimage rotated based on the rotation of the first image; and causing theidentified supplemental content to be presented in the portion of thedisplay that does not include the portion of the first image rotatedbased on the rotation of the first image.
 8. The method of claim 7,further comprising detecting an eye axis of the viewer.
 9. The method ofclaim 7, further comprising receiving an image of the viewer from acamera.
 10. The method of claim 7, further comprising: incrementing therotation of the first image to a new rotation of the first image; andgenerating another display image that includes the at least a portion ofthe first image or a subsequent image rotated based on the new rotationof the first image.
 11. The method of claim 7, further comprisingdelaying generating the display image a predetermined amount of timeafter detecting a change in the orientation of the viewer.
 12. Themethod of claim 7, wherein the supplemental data comprises at least oneof a program guide, controls for the at least a portion of the firstimage, and web content related directly or indirectly to the firstimage.
 13. A non-transitory computer-readable medium containing computerexecutable instructions that, when executed by a processor, cause theprocessor to perform a method for adjusting video displayed to a viewer,the method comprising: detecting an orientation of a viewer to a displaybased on at least two points associated with the viewer; determining,based on the orientation of the viewer to the display, a rotation of avideo image to be displayed on a display; determining a rotational limitfor the rotation of the video image; generating a display image fordisplay on the display, wherein the display image includes at least aportion of the video image rotated based on the rotation of the videoimage and subject to the rotational limit; determining a portion of thedisplay that does not include the portion of the video image rotatedbased on the rotation of the video image; identifying supplementalcontent to be presented in the portion of the display that does notinclude the portion of the video image rotated based on the rotation ofthe video image; and causing the identified supplemental content to bepresented in the portion of the display that does not include theportion of the video image rotated based on the rotation of the videoimage.
 14. The non-transitory computer-readable medium of claim 13,wherein the method further comprises detecting an eye axis of theviewer.
 15. The non-transitory computer-readable medium of claim 13,wherein the method further comprises receiving an image of the viewerfrom a camera.
 16. The non-transitory computer-readable medium of claim13, wherein the method further comprises: incrementing the rotation ofthe video image to a new rotation of the video image; and generatinganother display image that includes the at least a portion of the videoimage or a subsequent video image rotated based on the new rotation ofthe video image.
 17. The non-transitory computer-readable medium ofclaim 13, wherein the method further comprises delaying generating thedisplay image a predetermined amount of time after detecting a change inthe orientation of the viewer.
 18. The non-transitory computer-readablemedium of claim 13, wherein the supplemental data comprises at least oneof a program guide, controls for the at least a portion of the videoimage, and web content related to the at least a portion of the videoimage.